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FOREWORD 


This  publication  marks  one  of  the  first  attempts  by  the  Prevention  Branch,  Division  of 
Resource  Development,  National  Institute  on  Drug  Abuse,  to  share  information  gleaned  from 
projects  sponsored  by  demonstration  grants.  There  is  no  question  that  the  schools  are 
second  only  to  families  in  their  potential  for  influencing  the  drug-taking  behavior  of  the 
young.  In  this  report,  the  Southern  Regional  Education  Board,  an  educational  compact  of 
14  Southern  States,  has  crystallized  some  of  the  best  thinking  to  date  about  the  school’s 
roie  and  responsibility  in  the  area  of  drug  abuse  prevention.  This  is  one  of  a continuing 
series  of  tools  that  the  Prevention  Branch  is  making  available  to  assist  schools  in  their 
important  role  in  influencing  young  people.  While  this  report  is  of  necessity  limited,  it  is 
anticipated  that  information  will  be  added  as  new  developments  warrant  and  materials 
emerge  to  advance  the  state  of  the  art  of  drug  prevention  and  education. 
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INTRODUCTION 


This  publication  provides  some  guidance  to  alcohol  and  drug  education  program  adminis- 
trators by  clarifying  the  different  levels  of  evaluation  and  the  kinds  of  learning  that  can 
occur  at  each  level.  While  it  outlines  the  components  and  considerations  for  evaluation,  it 
does  not  define  a step-by-step  procedure.  In  short,  it  serves  as  a diet  rather  than  a recipe. 

A second  objective  for  the  publication  is  to  take  away  the  threatening  overtones  to  the  word 
“evaluation”  by  showing  the  benefits  to  be  gained  and  by  pointing  out  the  value  that  nega- 
tive findings  have  for  the  ultimate  success  of  a program. 

This  book  contains  four  sections:  Why  Evaluate?;  Guidelines  for  Impact  Evaluation; 
Suggestions  for  Process  Evaluation;  and  Evaluation  of  Drug  Programs  (synopses  of  some 
examples  of  evaluations  of  alcohol  and  drug  education  programs). 

The  section  of  the  book  dealing  with  impact  evaluation  was  developed  by  a task  force 
sponsored  by  the  Southern  Regional  Education  Board  project,  Enhancing  Drug  Education 
in  the  South.1  The  task  force  included  State-level  program  directors,  representatives  from 
Federal  agencies  concerned  with  alcohol  and  drug  education,  and  people  with  expertise  in 
program  evaluation. 


Xenia  R.  Wiggins 
Project  Director 
Enhancing  Drug  Education 
in  the  South 


'The  Project  was  designed  to  bring  together  State -level  alcohol  and  drug  educators  from  the  SREB  region 
in  problem-solving  conferences  and  task-oriented  workshops.  The  14  SREB  States  are  Alabama,  Arkansas,  Florida, 
Georgia,  Kentucky,  Louisiana,  Maryland,  Mississippi,  North  Carolina,  South  Carolina,  Tennessee,  Texas,  Vir- 
ginia, and  West  Virginia. 
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WHY  EVALUATE? 


Many  drug  education  programs  have  been  “crash”  programs,  created  by  public  panic  and 
pressure  on  public  officials  to  “do  something  quick”  about  the  drug  problem.  In  response 
to  what  was  defined  as  a crisis  situation,  we  hurried  to  develop  informational  programs  under 
the  assumption  that  information  would  change  behavior.  In  our  rush  we  frequently  began 
with  inconsistent  objectives  and  poorly  thought  out  plans  for  monitoring  the  results.  Where 
evaluation  did  occur,  it  indicated  that  informational  programs  were  not  effective,  even  where 
information  was  objective  and  scare  techniques  were  avoided.  Consequently,  drug  education 
is  being  called  to  task.  To  receive  continued  support,  program  directors  must  offer  new  and 
clear  approaches  to  drug  education;  they  must  demonstrate  to  their  funding  agency’s 
satisfaction — and  ultimately  to  the  public’s  satisfaction — that  these  new  directions  do 
contribute  to  a decreased  drug  problem. 

In  most  cases  program  directors  are  short  in  evaluation  skills.  Their  “we  have  to  do 
something  now”  response  to  pressure  for  accountability  leads  to  a variety  of  activities,  from 
highly  sophisticated  research  to  simple  activity  counts,  all  being  submitted  under  the  label 
of  “evaluation.”  Too  often,  the  variety  of  approaches  to  evaluation  that  lies  between  these 
two  extremes  is  not  considered — the  strengths  and  limitations  of  different  levels  of  com- 
plexity in  evaluation  are  not  explored  to  help  program  directors  determine  the  level  that  best 
meets  their  program  needs  and  budget.  And  too  often,  the  reasons  for  evaluation  and  the 
importance  of  evaluation — beyond  program  justification — are  not  realized. 

WHAT  IS  EVALUATION?  (Conceptual  Issues  and  Definitions) 

The  range  of  viewpoints  on  the  reasons  for  and  methods  of  evaluation  varies  greatly 
among  program  administrators.  Some  think  that  “good”  evaluation  requires  formal  research 
procedures,  while  others  use  personal  opinions  and  subjective  judgments.  Although  personal 
opinions  may  not  be  accurate  evaluations,  on  the  other  hand  it  is  not  necessary  to  conduct 
highly  complex,  sophisticated  research  with  rigid  controls  and  precise  measurement  in 
order  to  gain  reliable  information  for  evaluating  program  effectiveness. 

While  evaluation  often  uses  research  techniques  for  data  collection  and  analysis,  there 
are  differences  in  the  objectives  of  evaluation  and  research;  consequently,  the  scientific 
design  and  the  degree  of  scientific  precision  required  are  different.  Evaluation  is  conducted 
to  provide  reliable  information  to  program  directors  as  a basis  for  decisionmaking.  Constant 
feedback  of  findings  into  the  program  is  needed  so  that  changes  for  the  good  of  the  program 
and  its  clients  can  be  made  as  soon  as  possible.  It  is  not  usually  necessary  to  have  the  tight 
controls  on  data  collection  that  are  required  for  the  sophisticated  statistical  analyses  often 
employed  by  research.  This  does  not  imply  that  evaluation  can  be  sloppy,  but  rather  suggests 
that  it  should  be  more  concerned  with  accurately  reflecting  trends  and  changes  (as 
influenced  by  the  program)  in  the  target  population,  rather  than  developing  new  research 
tools  and  testing  theoretical  hypotheses. 

While  the  point  intended  here  is  to  have  administrators  recognize  that  evaluation  need 
not  be  extremely  costly  and  complicated,  we  do  not  want  to  imply  that  program  directors 
should  go  to  the  other  extreme.  Personal  judgments  and  simple  number  counts  of  activities 
sponsored,  material  distributed,  people  involved,  etc.,  do  have  their  place  in  program 
evaluation,  but  their  limitations  should  be  recognized. 
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It  is  important  to  clarify  two  different  dimensions  of  evaluation.  One  is  “impact 
evaluation,”  or  a measure  of  changes  that  took  place  in  the  target  population.  The  other  is 
“process  evaluation,”  a description  of  the  target  population  and  the  techniques,  methods, 
etc.,  which  were  used.  For  example,  a drug  educator  needs  to  know  how  many  and  what 
kinds  of  people  are  being  reached  by  his  program  in  order  to  determine  if  there  are 
segments  of  the  population  who  should  be  involved  but  are  not  currently  involved  (process 
evaluation).  Simple  logs  and  number  counts  will  give  him  this  kind  of  information  but  will 
not  tell  him  anything  about  the  effect  of  his  program  on  the  audience’s  drug  use  decisions 
and  behavior  (impact  evaluation).  Each  type  of  evaluation  has  its  contributions  and  its 
limitations,  and  administrators  need  to  understand  these  in  order  to  decide  what  level  of 
evaluation  is  needed  to  provide  them  the  information  they  need. 

Cost  is  a major  factor  in  deciding  the  complexity  of  evaluation.  The  more  sophisticated 
the  tools  and  the  more  complex  the  evaluation  design,  the  more  costly  evaluation  will  be. 
Considering  the  objectives  of  evaluation  we  have  discussed,  there  comes  a point  where 
doing  more  sophisticated  evaluation  does  not  sufficiently  increase  the  reliability  and  validity 
of  the  results  to  make  it  worth  the  added  costs.  Later  in  this  publication,  some  techniques  for 
reducing  costs  are  listed. 

REASONS  FOR  AND  AGAINST 

An  honest  answer  to  “why  do  we  do  evaluation?”  is  very  often  “to  justify  the  continued 
existence  of  the  program.”  In  too  many  cases,  our  primary  aim  is  to  prove  that  a job  has 
been  done  well  in  order  to  secure  more  funding,  protect  jobs,  etc.  The  possibility  of 
negative  findings,  and  hence  the  possible  loss  of  support,  makes  evaluation  a threat. 

Even  where  findings  are  favorable,  evaluation  cannot  guarantee  continued  funding,  for 
there  are  often  extraneous  political  forces  that  operate  on  decisions  to  continued  or 
discontinue  programs.  Also,  the  time  required  for  meaningful  evaluation  is  often  incom- 
patible with  funding  practices.  Funding  agencies  may  want  to  know  points  of  program 
impact  that  cannot  be  known  until  long  after  the  refunding  decision  has  been  made. 
Consequently,  the  “evaluation”  that  many  programs  send  to  funding  agencies  seldom 
qualifies  as  “good  evaluation.”  Since  evaluation  based  on  a funding  motive  cannot  guaran- 
tee funding  and  does  not  always  produce  useful  information,  administrators  would  do  well 
to  explore  other  reasons  for  conducting  evaluation  and  benefits  to  be  gained. 

The  task  force  explored  a number  of  reasons  which  support  evaluation,  along  with 
reasons  frequently  given  against  evaluation  (commonly  referred  to  as  the  yes,  but  . . . 
syndrome).  They  are  categorized  as  follows: 

Why  Evaluate? 

Humanistic 

1.  Helps  you  know  if  you  are  meeting 
target  needs 

2.  Money  spent  on  services  may  be  wasted 
if  you  have  no  way  to  know  if  you  are 
doing  what  you  intended  to  do. 

Administrative  Reasons 

1.  You  need  to  demonstrate  that  what  you  1.  No  money 

are  doing  are  steps  to  what  you  hope  to  2.  No  staff 

accomplish. 

2.  Evaluation  is  important  for  future  3.  No  time 

program  planning — knowing  what  works 
and  what  doesn’t  work  or,  all  else  being 
equal,  which  approach  is  cheaper. 


Why  Not  To  Evaluate? 

Reasons 

1.  You  can’t  quantify  people 

2.  Money  spent  on  evaluation  means  less 
money  spent  on  services. 
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Sophisticated  Reasons 

1.  You  can  measure  well-defined  objectives.  1.  Program  evaluation  “can’t  be  done.” 

It  may  be  too  early  to  evaluate  program  You  can’t  isolate  the  influence  of  your 

success  in  terms  of  long-range  goals,  but  program  from  the  influence  of  other 

you  can  demonstrate  that  immediate  social  forces  in  operation  at  the  time, 

objectives  are  in  the  direction  of  long- 
range  goals. 

If  the  purpose  of  evaluation  is  to  develop  a better  program,  then  negative  findings  are 
significant  and  useful  rather  than  threatening.  The  program  may  learn  its  weaknesses,  but  it 
also  learns  its  strengths.  It  has  an  advantage  that  it  did  not  have  before  in  knowing  what 
areas  need  revision  and  more  concentrated  attention  that  will  further  the  probability  of  a 
successful  program.  New  strategies  can  be  tried,  evaluated,  revised,  etc.,  in  a continued 
learning  process.  However,  if  the  only  objective  of  evaluation  is  to  meet  funding  guidelines, 
there  is  not  enough  time  in  a funding  period  for  this  valuable  learning  experience  to  occur. 
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GUIDELINES  FOR  IMPACT  EVALUATION 


Impact  evaluation  cannot  show  to  what  extent  a single  program  is  responsible  for  bringing 
about  a specific  change  because  it  cannot  separate  program  influence  from  the  influences 
of  other  current  trends.  However,  it  does  demonstrate  whether  or  not  the  expected  change 
has  occurred.  Discovering  that  a desired  change  is  not  taking  place  under  existing 
strategies  is  a significant  finding.  This  has  certainly  been  the  case  in  drug  education  where 
evaluation  has  shown  that  informational  approaches  not  only  do  not  reduce  drug  use  but 
may  stimulate  drug  use  or  alternative  drug  use.  Without  such  studies,  drug  education  might 
have  continued  on  its  earlier  path  at  great  expense  and  waste  of  time.  If  there  had  been 
more  evaluation  of  alcohol  education  and  anti-smoking  campaigns,  which  used  the  same 
strategies  that  drug  education  has  used,  and  if  the  findings  had  been  widely  distributed, 
perhaps  drug  educators  would  not  have  had  to  rediscover  the  fact  that  information  has  little 
influence  in  personal  decision  to  use  or  not  use  drugs  where  emotional  and  social  pressures 
are  high. 

IN-HOUSE  VS.  INDEPENDENT  EVALUATION 

An  important  early  decision  for  program  directors  is  whether  or  not  to  make  impact 
evaluation  an  in-house  function  of  the  program.  To  some  degree,  this  decision  is  influenced 
by  the  extent  of  in-house  capabilities  and  funding  available.  However,  there  are  additional 
issues  which  should  be  considered. 

There  are  arguments  for  either  choice,  and  the  director  faces  a tough  decision  in  deter- 
mining which  “trade-offs”  he  can  accept.  In-house  evaluation  is  usually  less  expensive  than 
independent  evaluation.  At  the  same  time,  it  runs  a greater  risk  of  being  biased.  If  the 
evaluator  (or  evaluation  team)  is  a member  of  the  program  staff,  he  has  an  investment  in 
seeing  the  program  succeed,  not  only  in  the  psychological  sense  of  wanting  to  be  part  of  a 
“successful”  program,  but  especially  in  the  practical  sense  of  wanting  to  secure  his  job. 
One  suggestion  for  reducing  the  risk  of  bias  in  in-house  evaluation  is  to  submit  the 
evaluation  for  outside  peer  review. 

The  program  evaluator  who  is  a member  of  the  program  staff  should  be  relatively  free 
from  responsibility  for  program  development.  Although  evaluation  staff  should  maintain  close 
contact  with  program  development  staff,  it  is  generally  best  not  to  assign  the  two  functions 
to  the  same  person. 

When  evaluation  is  independent  of  the  program,  there  is  still  a need  for  minimum 
in-house  capabilities  to  judge  the  quality  of  services  being  purchased  and  to  effectively 
use  the  evaluation  findings  in  program  decisions.  Someone  on  the  staff  should  know  about 
existing  services — available  from  agencies  in  the  same  system,  for  example — that  would 
help  reduce  the  costs  of  evaluation. 

In  short,  whether  evaluation  is  done  in-house  or  independently,  there  are  minimum 
in-house  skills  needed  by  program  staff  who  will  be  working  closely  with  evaluation  staff. 

Program  staff  need  to  be  familiar  with  evaluation  procedures  and  rationale  to  the  extent 
that  they  can: 

1.  Understand  and  use  evaluation  jargon 

2.  Work  with  evaluation  staff  in  translating  program  objectives  into  measurable  terms 
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3.  Make  informed  decisions  concerning  the  level  of  sophistication  in  evaluation  design 
needed  to  get  reliable  answers  to  questions  asked 

4.  Judge  the  competence  of  evaluation  staff  and  the  cost  of  the  total  evaluation  package 

5.  Use  the  evaluation  findings  for  program  change 

PLANNING  PROGRAM  OBJECTIVES/EFFECTIVE  USE  OF  A CONSULTANT 

The  evaluation  person,  whether  on  program  staff  or  independent,  should  serve  as  an 
“alter  ego”  to  the  program  director.  He  should  be  free  to  make  statements  such  as  “You 
can’t  do  that  in  2 days.  Change  your  objectives  or  change  your  timespan.”  And  “You  can’t 
measure  mental  health;  you  can  measure  divorce  rate,  suicide  rate,  admissions  to  mental 
health  centers,  etc.,  but  these  measures  are  not  pure  measures  of  mental  health.”  He  should 
also  be  free  to  suggest  new  questions  that  might  be  considered  for  evaluative  research. 

For  example,  evaluation  of  drug  education  programs  has  consistently  looked  at  changes  in 
drug  knowledge,  attitudes,  and  use  rates.  This  is  old  information  now,  and  spending  more 
time  and  money  looking  at  the  same  questions  would  probably  be  a waste.  Of  course,  the 
questions  asked  by  evaluation  tie  in  closely  to  the  objectives  of  the  drug  education 
program.  In  the  past,  drug  education  has  aspired  to  increase  knowledge  about  drugs  on 
the  assumption  that  more  knowledge  will  result  in  reduced  use.  Drug  education  is  changing 
now  in  the  direction  of  objectives  for  total  behavior,  not  just  drug-using  behavior,  and 
evaluative  questions  will  be  different. 

If  we  are  going  to  base  our  “drug  education”  programs  on  the  belief  that  drug-using 
behavior  cannot  be  separated  from  other  behavior,  we  should  be  looking  at  total  lifestyles 
and  learning  more  about  the  behavior  patterns  of  people  who  do  not  become  seriously 
involved  in  drug  use.  Rather  than  looking  again  at  drug  knowledge  and  attitudes,  there  are 
other  questions  concerning  individual  behavior  which  would  give  us  better  information 
about  the  success  of  programs  in  drug  abuse  prevention: 

1.  What  is  the  strength  of  interpersonal  and  intrapersonal  skills? 

2.  What  is  the  capacity  for  responsibility  and  independence? 

3.  What  is  the  extent  of  neurotic  maladjustment? 

4.  What  levels  of  anxiety  exist? 

5.  How  well  can  the  individual  internalize  and  manifest  behavior  control? 

6.  What  is  the  extent  of  alienation? 

Whenever  possible,  the  evaluation  consultant  should  be  involved  from  the  beginning  of 
the  program  when  objectives  are  being  defined.  In  this  way,  objectives  can  be  specified 
according  to  measures  to  be  employed,  the  information  needed  can  be  clarified,  and  time 
schedules  can  be  established  to  insure  the  most  appropriate  times  possible  for  data  collec- 
tion. Evaluation  designed  and  conducted  late  in  the  program  operation  will  not  give  com- 
prehensive or  valid  information,  nor  will  it  allow  for  feedback  into  the  program  at  different 
stages  while  there  is  still  time  to  correct  problems. 

A program  director  may  choose  not  to  evaluate  every  aspect  of  his  program;  he  may 
have  objectives  that  are  immeasurable  but  important  to  the  program  because  of  the  sense 
of  direction  they  provide.  Any  objective  to  be  considered  for  evaluation  must  pass  two  tests: 
It  must  be  feasible  and  it  must  be  measurable. 

The  task  force  outlined  four  essential  components  to  include  in  defining  objectives 
for  evaluation: 

1.  It  must  contain  a description  of  the  target  population  (e.g.,  all  Washington  high  school 
students,  or  a random  sample  of  30  students  from  sophomore  health  education  classes 
at  Washington  High),  and  the  desired  outcome  to  be  measured. 

2.  It  must  contain  a statement  of  time  involved  or  the  “deadline”  by  which  the  desired 
outcome  should  occur.  The  time  limit  should  be  consistent  with  the  objective  (i.e.,  don’t 
set  a 3-month  deadline  for  an  outcome  that  can’t  possibly  occur  in  that  short  period) 
and  should  vary  for  objectives  set  for  different  stages  in  the  program. 
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3.  It  must  contain  an  explanation  of  “key  factors”  and  “key  indicators.”  For  example, 
alcohol  education  may  have  as  its  objective  to  reduce  social  problems  related  to 
drinking  alcohol.  One  key  factor  might  be  alcohol-related  automobile  problems.  Key  indi- 
cators under  this  would  be  rates  of  arrest  for  driving  under  the  influence,  and  alcohol- 
related  automobile  accidents.  Another  key  factor  could  be  alcohol-related  employment 
problems.  Key  indicators  would  be  absenteeism  resulting  from  alcoholism  and  industrial 
accidents  caused  by  drinking. 

4.  It  must  have  a rationale  which  explains  the  logic  underlying  the  choice  of  the  objective 
as  a step  toward  the  overall  goal.  This  will  help  avoid  a complete  change  of  objectives 
in  mid-program. 

Objectives  should  be  consistent  with  the  needs  of  the  community  for  which  the  program 
is  designed.  Program  directors  will  be  more  successful  in  defining  objectives  that  respond 
to  the  community  if  they  spend  time  with  members  of  the  target  population,  involve  them  in 
program  planning  and  talk  with  people  from  other  agencies  concerned  with  the  drug 
problem.  It  is  also  important  that  new  program  directors  take  into  consideration  the 
viewpoints  for  which  some  agencies  know  the  drug  problem  and  recognize  that  their  view- 
point on  community  needs  may  represent  only  a part  of  the  total  picture. 

The  evaluation  consultant  should  anticipate  negative  findings  and  be  able  to  work 
with  the  program  director  in  planning  alternative  strategies  and  in  understanding  the 
reasons  for  the  findings.  At  the  same  time,  the  program  director  should  let  the  evaluator 
know  in  the  beginning  what  his  degrees  of  freedom  are;  that  is,  how  much  programmatic 
change  can  be  tolerated  within  the  real  constraints  of  the  system. 

REDUCING  THE  COST  OF  EVALUATION 

Most  program  directors  would  like  to  have  more  information  about  their  program’s 
impact,  but  they  do  not  have  the  funds  or  staff  to  spare.  Staff  may  keep  simple  logs  of 
program  activities,  participant  responses,  etc.,  that  provide  valuable  information  at  little  cost, 
but  these  do  not  qualify  as  impact  evaluation.  There  are  some  ways,  however,  to  conduct 
more  sophisticated  program  evaluation  and  still  keep  costs  at  a minimum.  A number  of  com- 
ponents of  evaluation  costs  and  suggestions  for  reducing  them  are  outlined  as  follows: 

1.  Consultation:  Evaluation  expertise  may  not  exist  on  program  staff,  but  there  is  a good 
chance  that  it  exists  somewhere  within  the  system  of  which  the  program  is  a part.  For 
example,  school  systems  often  have  research  units.  If  it  is  necessary  to  go  outside  the 
system,  faculty  at  nearby  college  or  university  and,  particularly,  doctoral  candidates  in 
need  of  a dissertation  may  be  willing  to  design  and  write  the  evaluation  in  exchange  for 
publication  rights.  The  program  director  should  recognize,  however,  that  the  faculty 
person  has  his  own  agenda  and  should  take  measures  to  insure  that  it  does  not  interfere 
with  evaluation  needs. 

2.  Instrument  design,  duplication,  and  testing:  It  may  be  possible  to  get  help  from  research 
industries  in  designing  and  testing  instruments.  The  program  director  gets  the  tool  he 
needs  and  aids  the  research  industry  in  developing  a marketable  instrument  by  providing 
a test  sample  in  return.  Again,  college  based  research  programs  may  lend  a hand,  or 
State  psychological  organizations  may  have  a free  services  program. 

3.  Administering  questionnaires/ interviews:  One  good  place  to  look  for  volunteer  help  is  in 
social  science  classes  at  local  colleges.  Students  are  looking  for  an  opportunity  to  test 
some  of  the  skills  they  are  learning  in  class  and  may  need  such  experiences  as  part  of  a 
course  requirement.  Another  source  of  help  is  volunteer  organizations.  Groups  such  as 
Junior  League  often  have  members  with  the  expertise  needed  and  with  a desire  to  work 
with  a social  service  agency.  A number  of  drug  education  programs  have  called  on  such 
volunteer  groups  for  help  in  a variety  of  areas.  Short-term  training  can  produce  a group 
of  very  effective  workers. 
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4.  Coding  instruments/ keypunching:  Getting  raw  data  into  a usable  form  is  one  of  the  most 
costly  components  of  evaluation.  Again,  students  learning  the  skills  needed  and  volunteer 
groups  are  good  resources.  High  school  students  could  also  do  this  aspect  of  a special 
science  or  drug  education  program. 

5.  Computer  time:  Most  computer  centers  have  canned  programs  that  are  readily  adaptable 
to  the  data.  Computer  time  may  be  obtained  from  universities  at  little  or  no  cost.  Also, 
businesses  in  the  community  may  be  willing  to  donate  computer  time  and  programming 
help  as  a community  service.  It  is  important  to  know  in  the  beginning  what  computer 
will  be  used  and  basically  the  kind  of  program  needed  so  that  the  data  can  be  coded 
and  prepared  (put  on  cards  or  on  tape)  in  a manner  compatible  with  the  computer  system 
and  the  program. 

6.  Nonreactive  measures:  Nonreactive  measures  or  unobtrusive  measures  are  those  made 
on  naturally  occurring  events  (e.g.,  sale  of  drug  paraphernalia,  school  dropout  rates) 
and  therefore  can  be  obtained  from  existing  records  without  interacting  with  the  popu- 
lation under  study.  These  measures  are  generally  less  costly  since  it  is  merely  a matter 
of  collecting  existing  information  or  counting  observed  behaviors.  Nonreactive  or 
unobtrusive  measures  can  also  be  made  on  events  which  occur  naturally,  but  for  which 
no  records  are  kept.  For  example,  one  indicator  of  the  extent  of  use  of  alcohol  and  drugs 
can  be  obtained  by  counting  liquor  bottles,  beer  cans,  etc.,  found  in  garbage,  and  by 
monitoring  the  number  of  drug  paraphernalia  (syringes,  etc.)  found  in  the  area.  A person 
using  such  unobtrusive  measures  should  be  very  aware  of  all  the  possible  conditions 
influencing  his  “findings,”  so  that  he  will  not  jump  to  wrong  conclusions.  A nonreactive 
measure  needs  to  meet  the  following  criteria: 

a.  Variable — the  frequency  of  the  event  must  vary.  For  example,  school  dropout  rates 
are  not  constant  but,  rather,  fluctuate  over  time. 

b.  Feasible 

c.  Reliable — the  measure  consistently  reflects  the  same  outcome  in  repeated  studies 

d.  Face  validity — logically  relates  to  behavior  under  study 

e.  Ethical — since  subjects  are  uninformed,  their  confidentiality  should  not  be  violated 
through  reporting  and  identifying  individual  behavior.  Nor  should  they  be  studied 
under  potentially  embarrassing  circumstance  (e.g.,  no  one-way  mirrors  in  restrooms). 

It  is  also  possible  to  reduce  costs  by  using  available  data  if  it  provides  the  information 
needed  and  covers  a time  period  close  enough  to  the  ideal  time  period  to  make  it  useful. 
Very  often  the  same  population  (e.g.,  students  in  public  school  system)  is  inundated  with 
questionnaires  asking  basically  the  same  questions.  Much  time  and  money  could  be  saved  if 
agencies  would  coordinate  their  data  collection  or  would  borrow,  when  possible,  from  the 
information  already  collected  by  other  programs. 

This  discussion  does  not  cover  all  the  components  of  evaluation  costs  nor  the  ways  in 
which  costs  can  be  reduced.  A resourceful  administrator  with  a little  imagination  can  get 
an  evaluation  of  his  program  with  little  additional  costs  and  probably  no  extra  staff. 

If  the  program  director  has  evaluation  funds  and  chooses  to  contract  for  services,  he 
should  ask  for  proposals  with  budgets  so  that  he  will  know  what  he  wants  and  how  to 
recognize  it  when  he  gets  it. 

USE  OF  FINDINGS 

The  use  to  which  evaluation  results  can  or  will  be  put  depends  on:  (1)  the  point  during 
program  operation  at  which  results  are  available;  (2)  the  constraints  in  operation  that  may 
hinder  making  changes  recommended  by  the  evaluation  results;  and  (3)  the  decisions  of  the 
program  director  and  his  consultants. 

The  point  at  which  evaluation  results  are  available  to  program  directors  is  an  important 
factor  in  determining  the  use  of  findings.  To  some  extent  this  point  is  determined  by  the 
length  of  the  drug  education  program.  In  very  short  programs  (even  a program  lasting 
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several  months  may  be  too  short),  there  is  not  enough  time  available  to  evaluate,  make 
suggested  program  revisions,  and  reevaluate.  In  such  cases,  program  directors  are  limited 
to  feedback  which  occurs  only  after  the  program  has  ended.  Ongoing  programs  or  programs 
of  long  duration  have  the  option  to  provide  for  periodic  evaluative  feedback.  Some  would 
argue  that  periodic  feedback  destroys  the  opportunity  to  measure  the  effectiveness  of  the 
original  program  by  causing  continuous  program  revision.  The  point  to  remember  is  that  an 
objective  of  evaluation,  whenever  possible,  is  to  provide  a good  information  basis  from  which 
to  make  programmatic  decisions  and  improvements.  If  evaluation  results  are  available  only 
at  the  end  of  the  program,  they  cannot  serve  this  function  except  to  guide  new  programs 
which  may  be  trying  to  accomplish  similar  objectives. 

When  evaluation  is  built  into  every  step  of  program  development  and  feedback  is 
continuous,  the  program  director  has  access  to  findings  at  any  point  and  can  use  them  as  a 
basis  for  deciding  whether  or  not  the  next  step  should  be  carried  out  as  first  planned. 
Continuous  evaluation  can  serve  an  ongoing  administrative-management  function. 

The  program  director  needs  to  recognize  that  continuous  feedback  from  evaluation  may, 
at  some  point,  call  for  a program  revision  including  new  objectives  that  would  require 
reevaluation  and  a need  to  collect  new  kinds  of  data. 

Evaluation  findings  would  have  greater  value  (beyond  their  use  to  the  individual 
program)  if  the  final  report  and  perhaps  even  significant  periodic  reports  were  shared  with 
program  directors  or  program  planners  who  could  benefit  from  the  learning  experience.  Dis- 
tributing the  information  through  publications  or  newsletters  generally  does  not  work  unless 
it  is  accompanied  by  a planned  followup  which  encourages  the  recipient  to  read  the 
material  before  filing  it  away.  The  task  force  suggested  bringing  program  planners  together 
in  a workshop  to  go  over  what  has  been  learned  through  evaluation  and  how  it  can  be 
applied  to  participants’  programs.  Such  a workshop  should  be  closely  followed  by  technical 
assistance  to  help  program  directors  implement  new  strategies,  etc.,  suggested  by  evaluation. 

COST-EFFECTIVENESS  EVALUATION 

Impact  evaluation  which  does  not  include  cost  effectiveness  leaves  out  a major 
component  necessary  to  future  program  planning.  While  it  is  important  to  learn  how  close 
a program  has  come  to  reaching  its  goal,  it  is  equally  important  to  know  the  cost  of  getting 
there.  If  the  cost  of  duplicating  or  continuing  a successful  program  is  prohibitive,  the  fact 
that  the  program  was  successful  is  of  little  consequence.  Cost  effectiveness  is  of  particular 
importance  to  funding  agencies.  If  two  programs  are  equally  effective,  the  funding  agency 
will  naturally  choose  to  support  the  program  which  operates  most  efficiently.  The  objective 
of  funding  agencies  is  to  cut  down  program  costs  as  much  as  possible  without  seriously 
sacrificing  program  effectiveness. 

The  general  guidelines  listed  below  offer  suggestions  for  planning  cost-effectiveness 
evaluation: 1 

1.  Perhaps  the  most  important  point  (and  the  first  step)  in  planning  cost-effectiveness 
evaluation  is  deciding  the  unit  of  service  or  output  to  be  measured.  Is  cost  to  be 
measured  “per  student,”  “per  student  hour,”  or  by  some  other  unit  of  measure? 
(“Student”  is  used  here  to  refer  to  a member  of  the  target  population.  It  is  not  restricted 
to  “student”  in  the  traditional  sense  of  the  word.)  Reporting  cost  for  the  total  program  is 
meaningless  because  it  does  not  allow  comparability  between  programs,  which  is  a major 
reason  for  cost-effectiveness  evaluation.  Nor  does  it  indicate  what  the  cost  covers.  The 
primary  criterion  for  selecting  the  unit  of  service  is  that  it  be  measurable. 

2.  Another  method  which  allows  comparability  between  programs  is  computing  cost  by 
program  function  (e.g.,  administration,  teaching,  public  information,  consultation  to  other 
programs,  etc.).  In  this  way,  any  services  (e.g.,  clearinghouse  function  or  public  infor- 

1 These  suggestions  are  adapted  from  “Cost  Analysis  System  for  Day  Care  Services,”  Southeastern  Day  Care 
Bulletin  No.  3,  Southern  Regional  Education  Board,  September  1971. 
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mation)  offered  by  one  program  but  not  offered  by  another  can  be  factored  out  in 
comparing  the  costs  of  the  two  programs.  All  expenses,  including  personnel,  are  listed 
by  function.  Where  expenses  apply  to  more  than  one  function,  they  are  apportioned 
according  to  the  best  estimate  possible.  For  example,  staff  members  such  as  the  director 
will  have  responsibility  in  more  than  one  program  area.  In  this  case  their  salaries  are 
distributed  under  each  function  according  to  the  proportion  of  time  they  estimate 
spending  in  that  function. 

3.  Donated  costs  (goods  and  services)  should  be  computed  and  included  to  give  an 
accurate  estimate  of  the  real  cost  of  duplicating  the  program.  Costs  should  be  computed 
only  for  the  donated  (or  reduced  costs)  goods  and  services  that  the  program  would  have 
purchased  otherwise.  Some  resources  may  be  free  of  charge  but  would  not  be  computed 
as  donated  because  they  are  readily  available  to  members  of  the  community  and  are 
not  provided  at  a special  time  or  place  for  the  drug  education  program. 

4.  An  important  decision  in  cost  analysis  is  deciding  the  time  interval  for  which  costs  are 
computed.  For  short-term  drug  education  programs,  this  decision  is  not  necessary  since 
costs  would  be  figured  from  program  start  to  program  end.  For  continuing  programs,  it 
is  important  to  choose  a time  interval  that  coincides  as  closely  as  possible  with  affiliated 
agencies  which  may  be  involved  in  the  bookkeeping  process.  The  same  interval  should 
be  followed  for  the  succeeding  periods  of  cost  accounting. 

5.  It  is  necessary  to  add  the  cost  of  equipment  purchased  to  obtain  the  total  cost  of  the 
drug  education  program.  The  major  reason  for  making  a distinction  between  supplies  and 
equipment  is  to  allow  for  depreciation  of  equipment.  Separating  equipment  cost  from 
supply  cost  permits  differentiating  between  costs  that  include  the  full  price  of  equip- 
ment (the  out-of-pocket  actual  expenditures)  and  the  costs  that  include  only  a yearly 
depreciation  allowance  for  equipment  (the  true  cost).  The  “Internal  Revenue  Service 
Depreciation  Guidelines  and  Rules”  is  helpful  in  determining  depreciating  rates  for 
various  kinds  of  equipment,  including  major  renovations  to  buildings. 


Itemized  Record — Donated  Services2 
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"Adapted  from  appendix  II  of  “A  Cost  Analysis  System  for  Day  Care  Services,”  Southeastern  Day  Care  Bulletin  No.  3,  Southern  Regional  Education  Board, 
September  1971. 


Itemized  Record— Donated  Goods3 
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''Adopted  from  appendix  I of  “A  Cost  Analysis  System  for  Day  Care  Services,”  Southeastern  Day  Care  Bulletin  No.  3,  Southern  Regional  Education  Board, 
September  1971. 


SUGGESTIONS  FOR  PROCESS  EVALUATION 


In  the  previous  section  we  were  concerned  primarily  with  impact  evaluation  or  measuring 
the  effect  of  an  alcohol  or  drug  education  program.  This  section  concerns  process  evaluation 
or  “What  was  done  to  whom’’  to  produce  what  effect.  There  are  some  simple  measures 
that  help  program  directors  monitor  program  progress  and  do  not  require  much  expense  or 
staff  time.  An  important  objective  of  such  records  is  to  keep  track  of  what  is  going  on  in  the 
community  (critical  incidents)  that  might  affect  program  results. 

DAILY  LOGS 

Alcohol  and  drug  education  program  directors  often  ask  their  staff  to  keep  daily  logs  of 
activities,  regardless  of  how  trivial  they  might  seem  at  the  time.  The  logs  usually  contain 
descriptions  of  activities,  people  contacted,  their  reactions,  impressions,  etc.  Such  descrip- 
tions can  help  staff  recognize  obstacles  to  program  implementation,  monitor  progress  in 
winning  support  from  other  community  agencies,  and  monitor  problem  resolution.  They 
provide  insight  into  changes  in  institutional  structures,  give  a picture  of  attitudes  of  people 
in  key  postions  and  also  indicate  the  amount  of  staff  time  required  for  different  program 
activities.  The  following  is  an  example  of  a log  entry  and  the  kinds  of  problems  it  can 
highlight: 

“August  25 — Escorted  by  high  school  principal,  visited  County  Health  Department, 
Community  Development  Center,  hospital,  President  of  School,  State  Senator.  Being 
introduced  in  this  way  will  prove,  I think,  to  be  a distinct  advantage.  Evidently  the 
principal  is  a respected  citizen  and  leader  in  the  community.  Talked  with  Episcopal 
rector.  Very  status  conscious.  Pictures  alcohol  attitudes  in  (community)  very  narrow. 
Felt  that  a major  obstacle  for  me  would  be  students  wanting  to  know  whether  I drink. 
Said  he  thought  I should  realize  that  I was  going  to  lose  effectiveness  with  one  group 
or  another.” 

In  addition  to  daily  logs,  some  program  directors  may  ask  their  staff  to  submit  written 
annual  or  quarterly  reports  covering  activities,  impressions,  evaluations,  and  suggestions. 

EVALUATION  CONFERENCES 

Program  directors  can  learn  a good  deal  about  problems  their  staffs  are  facing  and  their 
staffs’  success  in  resolving  them  through  periodic  staff  meetings.  Some  alcohol  and  drug 
education  programs  have  made  effective  use  of  year-end  evaluation  conferences.  Local 
citizens  and  outside  consultants  are  invited  to  react  to  the  program’s  activities.  Often,  new 
directions  and  emphases  will  emerge  from  such  conferences.  They  provide  an  opportunity 
for  input  from  the  population  being  served. 

DESCRIPTIVE  EVALUATIONS 

There  are  a number  of  simple  records  that  describe  program  activities,  who  was 
involved,  etc.  These  include: 

1.  Records  of  the  activities  sponsored  by  the  program  and  the  number  and  categories  of 
people  involved  in  each.  This  helps  show  where  program  priorities  lie  and  where  gaps 
exist  in  the  population  for  which  the  program  was  designed. 
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2.  Records  of  staff  time  spent  in  initiating  and  planning  activities.  This  indicates  to  the 
program  director  how  effectively  his  staff  is  being  utilized  and  what  proportion  of  time  is 
devoted  to  various  program  functions.  If  the  program  has  planned  times  for  staff 
functions  to  change  (e.g.,  an  objective  of  the  program  may  be  to  train  people  in  the 
community  to  take  over  and  continue  the  program  so  that  staff  functions  would  appropri- 
ately change  from  “drug  educator”  to  “consultant.”),  such  records  will  indicate  whether 
or  not  that  change  is  occurring. 

3.  General  information  requests  (requests  for  information  about  the  program) 

4.  Requests  for  publications  and  other  materia! 

5.  Records  of  speaking  engagements,  meetings  attended,  etc. 

6.  Records  of  media  coverage  of  the  program 

7.  Records  of  requests  for  consultations  (with  continued  updates  as  to  what  has  occurred) 

All  of  the  above  simple  measures  help  to  show  the  program’s  visibility. 

CONTENT  EVALUATION 

It  is  a good  idea  to  evaluate  workshops  and  other  educational  efforts  sponsored  by  the 
program.  In  addition  to  descriptive  information  mentioned  earlier,  program  directors  may 
want  to  measure  the  impact  of  workshops  upon  participants  through  brief  subjective 
assessments  of  topics,  formats,  instructors,  teaching  procedures,  etc.,  which  ask  participants 
to  identify  positive  and  negative  aspects. 

SPECIAL  STUDIES  OR  SAMPLE  STUDIES 

In  contrast  with  ongoing  evaluation,  special  studies  are  conducted  for  a limited  amount 
of  time.  For  example,  program  directors  may  ask  their  staff  to  keeo  very  careful  records 
of  their  time  for  a 1-week  period.  Of  course,  there  is  the  possibility  that  setting  aside  a 
special  time  of  study  will  influence  the  way  in  which  staff  spend  their  time  and  may  mean 
that  the  results  do  not  truly  reflect  the  usual  staff  patterns.  Another  example  is  to  closely 
monitor  telephone  calls,  letters,  etc.,  for  a given  period  of  time  to  study  the  kinds  of  reauests 
for  consultation,  material,  etc.,  that  are  being  made  of  the  program. 
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RECORD  OF  CONTACTS  AND  MEETINGS 


Individual 


Date  of 
contact 
or  mtg. 


recording  this  information 


Project  name  and  State 


In-  Out- 

coming  going 


Group,  organization,  or 
person  calling  meeting 
or  with  whom  contact 
was  made 


Major  purpose  or 
subject  of  contact 
or  meeting 
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EVALUATION  OF  DRUG  PROGRAMS 


The  descriptions  or  synopses  following  each  study  are  not  meant  to  be  judgmental  of  the 
program;  rather,  they  are  impartial  descriptions  written  for  the  benefit  of  persons  planning 
evaluations  or  similar  studies  of  drug  use,  attitudes,  knowledge,  etc. 

These  studies  were  not  chosen  on  the  basis  of  whether  they  were  considered  to  be 
“good”  or  “bad”  approaches  to  evaluation.  They  were  included  because  they  were  the  most 
accessible.  Among  the  many  evaluation  studies  identified  and  requested  by  the  Southern 
Regional  Education  Board  (SREB)  project,  these  are  the  ones  that  were  available  in 
published  form  or  sent  to  the  project  in  response  to  requests  for  copies. 

Exclusion  of  similar  studies  does  not  imply  that  the  ones  listed  here  are  the  only 
possible  approaches  to  evaluation  and  measurement. 

Nor  do  we  intend  to  imply  that  the  approaches  to  drug  education  evaluated  in  the 
included  studies  are  the  best  approaches  to  drug  education.  Most  of  these  evaluations  are 
designed  for  programs  based  on  the  information  model,  because  this  is  what  most  of  drug 
education  has  been  for  the  past  few  years.  Drug  educators  are  beginning  to  look  beyond 
information  dissemination  to  personal  skill  development  as  a more  effective  approach. 
Evaluations  of  these  newer  programs  will  look  very  different. 


Capone,  Thomas;  McLaughlin,  James;  and  Smith,  Frederick.  “Peer  group  leadership  program 
in  drug  abuse  prevention  1970-71,  academic  year.”  Journal  of  Drug  Education,  Vol.  3,  No. 
3,  Fall  1973.  pp.  201-245. 

(The  following  is  a direct  quote  of  the  “Summary  and  Conclusions”  of  the  article, 
pp.  242-243) 

“The  Peer  Group  Leadership  Program  in  Drug  Abuse  Prevention  developed  as  an 
attempt  to  utilize  the  impact  students  have  upon  each  other  in  order  to  prevent  and  combat 
drug  usage  among  high  school  students  in  New  York  City.  In  its  initial  year  (1970-71)  it 
extended  to  16  high  schools.  In  the  planning  phase,  designated  Phase  I,  student,  teacher,  and 
principal  participants  were  brought  together  to  plan  for  the  development  of  subsequent 
phases  of  the  program. 

“Phase  II  consisted  of  a five  day  training  and  orientation  period.  Intended  participants 
included  the  principal,  one  selected  teacher,  and  eight  selected  students  from  each  school. 
The  workshop  was  intended  to  foster  in  the  participants  skills  which  would  serve  them  as 
change  agents  within  their  student  bodies.  Students  were  also  to  formulate  plans  of  action 
at  this  time  which  could  be  implemented  upon  return  to  their  respective  schools. 

“The  purpose  of  the  present  evaluation  was  to  determine  the  extent  to  which  the  follow- 
ing program  objectives,  as  stated  in  the  program  proposal,  were  met: 

1.  To  develop  an  educational  program  which  has  as  its  focus  the  preventive  approach 
to  the  problems  of  teenage  drug  abuse. 

2.  To  enable  students  to  develop  for  themselves  the  skills  in  the  process  of  facilitating 
healthy  peer  group  relationships. 

3.  To  help  students  become  effective  leaders  and  change  agents  among  their  peers. 

4.  To  enable  program  students  to  formulate  objectives. 
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“The  evaluation  methodology  included  interviews  with  the  Director  of  the  Peer  Group 
Leadership  Program,  the  assistant  to  the  Chancellor,  and  the  Associate  Director  of  the  Bureau 
of  Planning  and  Innovation.  Questionnaires  were  administered  to  principals,  Teacher- 
Facilitators,  and  Peer  Group  Leaders.  In  addition,  the  evaluators  made  several  on-site  visits 
and  attended  several  training  sessions  in  order  to  observe  program  operations  firsthand. 

“On  the  basis  of  the  findings,  the  following  conclusions  can  be  drawn  regarding  the 
extent  to  which  the  stated  program  objectives  were  met: 

1.  An  educational  program  was  developed  which  had  as  its  focus  the  preventive  approach 
to  the  problems  of  teenage  drug  abuse.  The  nature  and  extent  of  program  development 
at  each  school  appeared  to  vary  considerably.  Approximately  17  percent  of  the  Peer 
Group  Leaders  indicated  their  group  had  done  nothing  in  this  regard.  However,  most 
students  appear  to  have  enthusiastically  engaged  in  this  phase  of  the  program. 

2.  Several  training  sessions  were  held  to  enable  students  to  develop  skills  in  facilitating 
healthy  peer  group  relationships.  Hopefully  the  training  sessions  would  lay  the  ground- 
work in  the  development  of  effective  leaders  and  change  agents  among  their  peers.  At 
present,  it  is  too  early  to  expect  evidence  of  the  extent  to  which  such  skills  have  been 
learned  and  successfully  applied. 

3.  It  does  not  appear  at  each  school,  that  students,  or  even  teacher  facilitators,  have 
formulated  a clear  statement  of  plans  of  action.  At  present,  as  might  be  expected  at  this 
early  stage,  only  general  formulations  of  program  objectives  and  activities  have  been 
made.  Hopefully,  movement  will  be  made  toward  developing  greater  specificity.  . . . 
Moreover,  it  would  appear  that  specific  activities  should  be  developed  with  the  aid  of 
consultants.  It  is  necessary  to  foster  cooperative  formulations  of  certain  common  goals 
among  schools  as  well  as  between  teacher-facilitators  and  students  at  a given  school.” 

Doerr,  Dale;  Kabat,  Hugh  F.;  Sheffield,  William  J.;  and  Skinner,  William  J.  “On  the  campus 

. . . drug  abuse  programs.”  Journal  of  the  American  Pharmaceutical  Association,  Vol. 

NS7,  No.  9,  September  1967. 

University  of  Minnesota 

To  stress  student  involvement,  the  American  Pharmaceutical  Association,  student 
chapter,  was  given  responsibility  for  planning,  publicity,  and  programming.  Sponsors  planned 
a 2-day  afternoon  conference.  Attention  was  given  to  timing — both  time  of  day  and  time  in 
the  academic  quarter — to  ensure  student  attendance. 

\A  film,  “Narcotics,  the  Decision,”  was  shown  the  first  day  with  the  main  presentations, 
a series  of  talks  followed  by  panel  discussion  to  be  given  the  second  day.  The  main  program 
featured  seven  half-hour  talks  followed  by  1-hour  panel  discussions. 

Program  sponsors  evaluated  the  conference  as  follows: 

1.  For  maximum  impact,  the  entire  program  should  be  presented  in  1 day.  This  conclusion 
was  based  on  the  drop  in  attendance  from  the  first  day  to  the  second  day. 

2.  The  1-day  program  should  be  no  longer  than  2Vi  hours.  It  was  felt  that  the  individual 
presentations  had  been  too  long  and  that  the  four  talks  on  use  and  abuse  could  have 
been  given  as  one  presentation. 

3.  A followup  program  for  residence  units  would  carry  the  message  to  smaller  groups  in 
a more  intimate  setting. 

4.  The  program  should  be  presented  every  other  year. 

University  of  Texas 

The  professional  Pharmacy  Fraternity  sponsored  a symposium  during  which  five 
speakers  each  made  20-minute  presentations.  A question-and-answer  period  followed.  The 
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program  was  conducted  in  the  evening  in  the  student  union  hall  and  reached  an  audience 
of  350-400  students.  (The  article  describes  publicity  measures  for  the  program.) 

Sponsors  were  disappointed  in  the  turnout  and  concluded  that  a successful  program 
“.  . . depends  on  well-known  speakers,  a topic  of  vital  interest,  and  heavy  advertising.” 

Butler  University 

The  professional  Pharmacy  Fraternity  sponsored  a series  of  group  discussions  in  housing 
units  climaxed  by  a formal  symposium.  The  total  program  lasted  1 week.  Each  housing  unit 
was  visited  by  a three-man  team  from  the  Pharmacy  Fraternity  that  led  group  discussions 
on  drugs,  their  use,  and  misuse.  The  symposium  speakers  included  a psychiatrist,  pharma- 
cologist, research  specialist,  and  inspector  for  the  Indiana  State  Board  of  Health.  The 
program  received  both  newspaper  and  TV  coverage  prior  to  the  symposium.  Planners  judged 
the  program  a success,  based  on  near-capacity  attendance. 

Johnson,  Barbara  B.  “A  junior  high  school  seminar  on  dangerous  drugs.”  The  Journal  of 
School  Health,  Vol.  38,  No.  2,  February  1968.  Pp.  83-87. 

Bancroft  Junior  High  School  in  the  San  Laredo  Unified  School  District,  California, 
conducted  a drug  education  program  for  eighth  and  ninth  grades.  The  program  objectives 
were  to  help  each  student  establish  a set  of  values,  look  at  himself  in  his  world,  look  at  peers 
who  are  an  important  part  of  his  life,  and  determine  what  he  would  do  when  faced  with 
decisions  about  drug  use. 

Stage  I:  English  classes  were  divided  into  six  or  seven  discussion  groups  with  a student 
leader  and  recorder.  Leaders  were  given  guidelines  such  as  encourage  all  students  to  talk, 
and  leaders  kept  records  anonymous.  Students  reacted  to  seven  statements  about  drugs, 
for  example,  ‘‘I  feel  I will  never  use  drugs,”  ‘‘I  would  like  to  know  these  things  about  drugs.” 

Stage  II:  A panel  of  experienced  persons  responded  to  students'  questions  in  two  separate 
assemblies  by  grade  level. 

Stage  III:  Film — “Narcotics — Pit  of  Despair” 

The  evaluation  is  based  on  students’  reaction  to  the  program  and  its  influence  on  their 
attitudes  and  on  the  subjective  feelings  of  the  program  planners.  The  report  does  not  indicate 
any  measure  of  student  attitudes  before  the  programs. 

Evaluation  in  English  Classes. — Sixty-five  percent  of  the  students  responded  favorably  to 
discussions  in  English  class.  Eighty-three  percent  of  eighth  graders  and  66  percent  of  ninth 
graders  felt  the  seminar  had  added  to  their  knowledge  of  drugs.  Approximately  the  same 
percentage  reported  that  the  seminar  had  influenced  them  against  making  drugs  a part  of 
their  lives. 

Evaluation  of  Panel — Students  were  disappointed  by  the  panel  because  their  questions  were 
not  answered,  discussion  was  not  related  to  the  immediate  environment,  and  panelist  spoke 
either  above  or  below  the  students.  Suggestions  for  improvement  include  having  students 
on  the  panel  or  having  students  interview  “experts”  either  in  front  of  their  peers  or  to  report 
back  to  their  peers. 

Evaluation  of  Film — Evaluation  reports  that  students  were  favorable  to  the  film. 

Lewis,  Jerry  M.;  Gossett,  John  T.;  Phillips,  Virginia  Austin.  “Evaluation  of  a drug  prevention 
program.”  Hospital  and  Community  Psychiatry,  April  1972. 

In  October  1969,  a questionnaire  surveying  drug  use  was  administered  to  all  junior  and 
senior  high  schools  in  the  Dallas  Independent  School  District.  Two  weeks  before  the  district- 
wide survey,  a 2-day  drug  abuse  prevention  program  was  held  at  one  of  the  larger  senior 
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high  schools.  The  questionnaire  to  be  used  in  the  district-wide  survey  was  administered 
three  days  before  the  program  to  a 10  percent  random  sample  (315  students)  at  the  school. 
A nearby  control  school,  similar  in  characteristics  to  the  experimental  school,  also  con- 
ducted a random  sample  survey  at  the  same  time.  Both  schools  were  then  resurveyed  during 
the  district-wide  survey. 

The  drug  program  in  the  experimental  school  was  presented  by  a private  organization. 
The  organization  trained  15  percent  of  the  student  body  as  group  leaders  to  conduct  small 
group  meetings  in  which  the  entire  student  body  participated.  Comparing  the  results  of  the 
before-and-after  questionnaires  for  the  two  schools  shows  that  . . of  the  32  drugs  or  groups 
of  drugs,  two  reflected  greater  usage  in  reported  usage  in  the  school  that  held  the  preven- 
tion program  than  in  the  control  school.  In  11  categories  both  schools  had  identical  rates  of 
change  in  reported  use,  generally  a 1 or  2 percent  increase.  In  19  categories  the  control 
school  had  a larger  increase  in  reported  use  than  did  the  school  with  the  prevention  program. 
Thus  while  the  antidrug  program  may  have  played  a part  in  the  very  slight  decline  in  usage 
in  19  categories  of  drugs,  it  failed  to  do  so  in  the  other  13.  Chi-square  analysis  of  this  result 
indicates  that  the  pattern  of  19  ‘successes’  and  13  ‘failures’  does  not  differ  significantly 
from  chance.” 

Mason,  Michael  L.  Drug  Education  Effects,  Final  Report.  Young  Adult  Service,  Gainesville, 
Florida. 

National  Center  for  Educational  Research  and  Development  (DHEWLOE) 

Grant  No.  OEG-4-71-0070 

(The  synopsis  below  is  quoted  from  ERIC  catalogue,  entry  no.  ED  071-011) 

“This  is  a research  project  that  was  intended  to  study  the  effects  of  a factual  drug 
education  program  on  the  attitudes  of  high  school  and  junior  high  students  toward  the  use 
of  psychoactive  drugs.  The  approximately  250  eighth  and  twelfth  grade  students  involved 
in  the  study  filled  out  a number  of  Questionnaires  designed  to  measure  a variety  of  their^ 
.attitudes  about  psychoactive  drugs  immediately  before  and  after  participating  in  a drug 
education  program  that  relied  heavily  on  the  presentation  of  known  facts  about  a variety  of 
drugs.  Analysis  of  the  data  indicated  that  the  students  learned  about  the  given  drugs  to  a 
highly  significant  degree,  their  curiosity  about  the  effects  of  “mind-expanding”  drugs  was 
increased,  and  they  exhibited  an  increased  tendency  to  deal  with  psychological  discomfort 
through  the  use  of  drugs.  At  the  same  time  they  reacted  more  favorably  toward  the  legaliza- 
tion of  marijuana  and  a reduction  of  penalties  for  drug  use,  and  less  favorably  toward 
present  emphasis  on  a legal  approach  to  the  use  of  drugs.  The  primary  conclusion  of  the 
study  is  that  drug  education  is  not  an  effective  means  of  suppressing  the  use  of  drugs. 
References  and  numerous  tables  are  included.” 

Monk,  Mary;  Tayback,  Mathew;  and  Gordon,  Joseph.  “Evaluation  of  an  antismoking  program 
among  high  school  students.”  Program  Evaluation  in  the  Health  Fields.  New  York: 
Behavioral  Publications,  1969. 

Two  comparable  senior  high  schools  in  Baltimore  participated  in  an  evaluation  of  an 
antismoking  program.  The  objective  of  the  program  was  to  persuade  students  to  stop  or  not 
to  begin  smoking.  Prior  to  beginning  the  campaign,  both  schools  completed  the  same 
questionnaire.  Students  in  the  experimental  school  had  26  exposures  to  the  antismoking 
campaign  over  7 months.  There  was  no  such  activity  in  the  control  school.  The  program 
concentrated  on  the  connection  between  lung  cancer  and  smoking  and  consisted  of  the 
following  basic  activities: 

1.  School-wide  assemblies  over  the  PA  system  involving  principal,  student  council, 
commissioner  of  health,  cancer  research  scientists  and  physicians. 
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2.  Sixteen  posters  displayed  in  home  rooms  at  semi-monthly  intervals. 

3.  Two  letters  from  commissioner  of  health  mailed  to  all  students. 

4.  Series  of  articles  in  school  newspaper. 

5.  Two  leaflets  distributed:  “Shall  I Smoke?”  (American  Cancer  Society),  and  “Why  Buy  a 
Pack  of  Trouble?”  displayed  in  gymnasium. 

In  addition,  the  school  physician  and  nurse  held  group  discussions  on  smoking  and 
health  with  nearly  all  10th  grade  students  in  groups  of  six  to  eight  boys.  Teachers  were 
asked  to  have  class  discussions  on  smoking  and  to  initiate  appropriate  classroom  projects. 

Findings 

late  spring,  both  schools  again  filled  out  the  same  questionnaire  they  had  completed 
in  the  fall.  Teachers  in  both  schools  completed  forms  indicating  how  many  students  had 
taken  part  in  classroom  discussions  or  projects  on  cigarette  smoking.  In  addition,  interviews 
were  conducted  with  95  students  representing  all  grades  and  courses  in  the  experimental 
schools. 

ATThe  questionnaire  measured  changes  in  rates  of  smoking  and  changes  in  attitudes 
toward  smoking.  The  antismoking  program  did  not  appear  to  affect  the  total  amount  of 
smoking  in  the  experimental  school.  Change  in  smoking  for  all  grades  in  both  schools  was 
negligible. 

The  article  gives  a detailed  report  of  comparisons  of  the  two  schools  in  students’ 
responses  to  eight  attitude  statements  about  smoking.  The  desired  change  at  the  experi- 
mental school  was  greater  than  at  the  control  school  for  only  two  statements,  and  for  only 
one  of  these  (“Smoking  is  dangerous  to  health”)  was  the  difference  between  schools 
significant. 

Although  the  questionnaire  did  not  show  it,  22  of  the  95  students  interviewed  stated  they 
smoked  less  in  the  spring  than  in  the  fall  (when  campaign  began);  however,  they  were  still 
regular  smokers.  The  objectives  of  the  interview  were  to  measure  students’  retention  of<d^ 
information  provided  in  the  campaign  and  to  measure  students’  reactions  to  the  program. 
Considering  the  theme  of  the  campaign  (the  relationship  between  cigarette  smoking  and  lung 
cancer),  only  a small  proportion  (16  percent)  of  students  mentioned  this  first  as  a way  smok- 
ing affects  health;  an  additional  5 percent  reported  “they”  say  smoking  causes  lung  cancer. 
Fifty-three  percent  gave  as  their  first  response  that  smoking  causes  shortness  of  breath. 

The  interview  showed  that  the  poster  had  been  by  far  the  most  noticed  part  of  the 
program.  Slightly  more  than  half  remembered  the  PA  system  lectures  and  less  than  half 
recalled  the  leaflets  or  mentioned  the  questionnaires. 

Conclusions 

The  article  suggests  several  points  to  consider  in  interpreting  the  findings: 

1.  A program  of  7-months’  duration  may  be  too  short  to  counteract  prosmoking  influences. 

2.  Greater  teacher  and  student  participation  may  have  increased  program  effectiveness. 

3.  The  program  was  conducted  at  a school  where  there  were  a large  number  of  regular 
smokers  who  are  more  likely  than  nonsmokers  to  resist  antismoking  education  programs. 

4.  Other  studies  have  shown  that  most  students  who  smoke  begin  a smoking  nabit  between 
ages  of  12  to  15.  Future  programs  should  concentrate  on  elementary  grades. 

5.  The  students  who  smoke  the  most  are  the  most  difficult  to  reach  through  regular  school 
activities. 

Norwalk,  Dorothy.  “Innovations  in  drug  education.”  Journal  of  School  Health,  Vol.  39,  No.  4, 
April  1969.  Pp.  236-239. 

A drug  education  program  for  Suitland  Senior  High  School,  Maryland,  was  planned  by  a 
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student  committee  composed  of  three  students  from  each  class.  The  stated  objectives  for 
the  program  were  to  provide  accurate  information  and  to  provide  a variety  of  viewpoints 
about  drugs,  drug  use,  drug  users  and  their  reactions  to  drugs. 

Teacher  preparation  consisted  of  viewing  a film  on  LSD  produced  by  the  U.  S.  Navy, 
receiving  a packet  of  materials,  and  hearing  an  illustrated  talk  by  a health  educator  from 
the  University  of  Maryland. 

Prior  to  the  program,  all  students  saw  the  LSD  film  in  small  groups.  Some  students  saw 
additional  films. 

The  entire  student  body  met  for  lectures  by  a leader  in  the  drug  field.  Presentation 
included  introduction  to  psychological,  physiological,  sociological,  and  legal  aspects  of  drug 
use  and  abuse.  Following  the  lecture,  students  returned  to  homerooms  for  1-hour  discussion 
sessions  with  the  teacher  and  resource  people  (doctors,  pharmacists,  nurses,  professors, 

BNDD  representatives,  lawyers,  and  education  directors  of  State  and  Federal  drug  programs). 

While  half  of  the  student  body  met  with  resource  people,  the  remaining  half  viewed 
three  films:  “Hide  and  Seek,”  “Narcotics:  Pit  of  Despair,”  and  “Drugs  and  the  Nervous 
System.”  The  next  hour,  the  two  groups  reversed  the  activities. 

Evaluation 

Immediately  following  the  program,  students  filled  out  a form  provided  by  their  teachers 
(1,044  students  responded). 

Results:  75  percent  rated  overall  value  of  program  as  excellent  or  very  good,  18  per- 

cent— good,  5 percent — fair,  2 percent — poor 
Lecture — 47  percent  rated  excellent  or  very  good, 

Films — 75  percent  excellent  or  very  good 
Discussion  groups — 68  percent  excellent  or  very  good 

The  teachers’  overall  value  of  the  program:  70  percent  rated  excellent  or  very  good,  28 
percent — good,  0 percent — Fair,  2 percent — poor 

The  consultant’s  overall  value  of  the  program:  70  percent  rated  excellent  or  very  good 
and  30  percent — good 

O’Keefe,  M.  Timothy.  “The  anti-smoking  commercials:  A study  of  television’s  impact  on 
behavior.”  Public  Opinion  Quarterly  35:242-248,  Summer,  1971. 

Self-administering  questionnaires  were  distributed  to  area  junior  and  senior  high  schools 
and  selected  freshmen  classes.  A total  of  621  responses  was  obtained.  In  addition,  a general 
population  survey  of  300  area  residents  was  conducted  by  telephone.  The  purpose  of  the 
survey  was  to  determine  the  effectiveness  of  a television  antismoking  commercial  in  persuad- 
ing people  to  stop  smoking  and  in  preventing  others  from  starting.  Results  of  the  survey  show 
that  commercials  appear  to  have  no  effect  on  the  majority  of  smokers  who  wanted  to  con- 
tinue. They  merely  provided  support  to  those  who  wanted  to  stop.  The  greatest  impact  was 
with  younger  people.  Smokers  recognized  the  health  hazards,  but  few  intended  to  stop  or 
cut  down. 

O’Rourke,  Thomas  R.,  “Assessment  of  the  effectiveness  of  the  New  York  State  drug  curricu- 
lum guide  with  respect  to  drug  knowledge.”  Journal  of  Drug  Education,  Vol.  3,  No.  1, 
Spring  1973.  Pp.  57-66. 

The  purpose  of  the  study  was  to  assess  the  effectiveness  of  a program  based  on  the 
revised  New  York  State  Curriculum  Guide  in  increasing  students’  knowledge  about  drugs, 
alcohol,  and  tobacco.  The  underlying  rationale  of  the  program  was  that  increased  knowledge 
would  assist  in  reducing  the  incidence  of  drug  abuse. 

The  study  was  conducted  with  a total  of  446  students  in  four  high  schools  in  Nassau 
County,  New  York.  The  two  experimental  schools  received  the  new  program,  and  the  control 


22 


schools  received  a regular  health  education  program  without  special  guidelines.  There  was 
no  pretest  given  to  any  of  the  schools.  Comparisons  are  made  between  the  control  and 
experimental  schools  on  a knowledge  test  given  to  both  groups  after  the  program.  The  author 
used  analysis  of  variance  to  measure  significant  degrees  of  difference  in  cognitive  achieve- 
ment between  the  two  sets  of  schools. 

The  experimental  group  scored  significantly  higher  in  knowledge  on  each  of  the  three 
subtests  (alcohol,  knowledge,  and  tobacco)  and  on  the  entire  test.  When  comparisons  are 
made  by  sex  of  respondent,  the  program  appears  more  effective  for  males  than  females. 

Pollock,  Marion  B.  “An  evaluation  instrument  to  appraise  knowledge  and  behavior  regarding 
use  of  stimulants  and  depressants.”  The  Research  Quarterly,  Vol.  39,  No.  3.  Pp.  662-7. 

“This  article  describes  the  process  by  which  a two-part  evaluation  instrument  was 
developed  to  assess  knowledge  and  behavior  regarding  the  use  of  stimulant  and  depressant 
substances.  A trial  instrument  was  administered  to  72  college-bound  high  school  seniors. 
Analysis  of  the  responses  was  used  to  construct  a refined  second  version  then  administered 
to  467  men  and  women  at  California  State  College  at  Long  Beach. 

The  article  draws  the  following  conclusions: 

“1.  There  was  a demonstrated  need  for  instruction,  and  for  an  instrument  that  could  be  used 
to  reveal  strengths  and  weaknesses  in  present  programs. 

2.  A test  was  constructed  that  would  (a)  evaluate  high  school  graduates’  knowledge  about 
the  use  and  effects  of  tobacco,  alcohol,  and  dangerous  drugs  and  narcotics,  and 

(b)  reveal  present  practices  in  use  of  these  substances. 

3.  Use  of  instructional  objectives  as  the  source  of  a representative  pool  of  items  was  dem- 
onstrated to  be  a valid  process  for  the  construction  of  improved  health  education  tests 

4.  The  test  satisfied  logical  and  statistical  criteria  of  validity,  reliability,  and  item 
discrimination.” 

(The  above  is  quoted  directly  from  the  article.) 

Selvin,  Julius  B.  “Effectiveness  of  programmed  materials  in  teaching  a secondary  school 
health  education  unit.”  The  Research  Quarterly,  Vol.  39,  No.  3.  Pp.  704-7. 

(The  synopsis  below  is  quoted  directly  from  the  article,  page  704.) 

“This  study  was  designed  to  determine  if  intrinsically  programmed  materials  and 
techniques  can  be  effectively  used  to  teach  a health  education  unit  to  secondary  school 
students,  and  to  compare  levels  of  achievement  resulting  from  the  utilization  of  programmed 
materials  as  opposed  to  traditional  classroom  methods.  Twelve  secondary  school  health 
education  classes  in  New  York  City,  involving  343  students,  were  divided  into  matched 
groups.  Branched,  self-teaching  materials  on  the  topic  of  alcohol  education  were  presented 
to  the  experimental  group  to  be  studied  at  home  for  a three-day  period  without  recourse  to 
classroom  instruction.  The  control  group  was  taught  the  topic  in  a 3-day  unit  utilizing  tradi- 
tional classroom  teaching  techniques.  The  results  of  the  achievement  test  administered  at 
the  conclusion  of  the  unit  indicate  that  branched  or  intrinsically  programmed  materials  and 
techniques  can  be  effectively  used  in  the  teaching  of  a secondary  school  health  education 
unit  and  that  in  this  particular  investigation  the  experimental  group  learned  significantly 
more  than  the  control  group.” 

A study  of  Student  Behavior  Changes  in  the  SPARK  Program  Intervention-Prevention 
Centers  conducted  by  the  New  York  City  Board  of  Education  SPARK  Program  for  the 
Addiction  Services  Agency  of  the  City  of  New  York,  April  1973. 

The  SPARK  Program  is  a drug  prevention  program  that  was  conducted  in  nine  New 
York  City  high  schools  with  indicators  of  high  need.  Some  of  the  program  activities  include 
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student-led  peer  group  programs,  identification  and  referral  of  drug  abusers  to  treatment, 
classroom  and  assembly  information-oriented  programs,  and  teacher  training.  The  primary 
activity  in  all  schools  was  counseling — individual,  rap  sessions,  ongoing  group  sessions — 
for  students  whose  behavior  patterns  indicated  a high  potential  for  drug  abuse.  The  study 
was  conducted  to  test  the  hypothesis  . . that  participation  in  SPARK  program  counseling 
would  alleviate  negative  behavior,  anti-social  behavior,  and  low  personal  achievement  which, 
the  literature  suggests  and  the  SPARK  program  assumes,  are  highly  correlated  with  drug 
abuse  factors  which  tend  to  foster  drug  abuse.” 

A random  sample  of  100  students  was  selected  from  the  ongoing  group  counseling 
sessions  in  each  of  the  nine  schools  during  the  September  1972-January  1973  semester 
(900  students,  total).  During  this  semester,  these  students’  records  were  examined  to  monitor 
absentee  rate,  referral  to  dean  for  disciplinary  purposes  by  classroom  teachers,  major 
subjects  failed,  overall  grade  point  average  and  conduct  ratings.  The  findings  of  this  exami- 
nation were  compared  to  records  for  the  same  students  during  the  September  1971  to 
January  1972  semester,  when  they  were  not  participating  in  the  SPARK  program.  ‘‘In  every 
category  studied,  the  aggregate  population  of  the  sample  . . . showed  a significant  change 
in  each  of  the  five  indicators  selected.”  Statistical  analysis  for  the  total  sample,  using  a 
two-tailed  t-test,  shows  that  all  changes  are  significant.  The  study  also  shows  findings  for 
each  individual  school  where  the  results  vary. 

Swisher,  John  D.,  et  al .,  “Four  approaches  to  drug  abuse  prevention  among  college  students.” 

Journal  of  College  Student  Personnel,  Vol.14,  No.  3.  Pp.  231-235. 

The  purpose  of  this  study  was  to  compare  the  relative  effectiveness  of  four  small  group 
approaches  to  drug  abuse  prevention  among  college  students: 

1.  Discussion  group — discussion  of  issues  and  problems  surrounding  drug  abuse  on  factual- 
informational  basis.  Group  leaders  were  instructed  not  to  let  the  group  become  feeling- 
oriented. 

2.  Relationship  counseling  group — students  were  free  to  explore  topic  in  any  manner  they 
chose.  Counselor  did  not  impose  his  opinions  on  group  but  served  to  help  members 
understand  and  express  themselves. 

3.  Reinforcement  counseling  group — included  counselor  and  two  college-age  role  models 
who  were  knowledgable  about  drug  culture  but  had  never  used  drugs.  They  were  to 
guide  discussion  toward  reasons  for  not  using  drugs  and  to  reinforce  students’  state- 
ments that  supported  their  not  becoming  involved  with  drugs. 

4.  Reinforcement  counseling  group — included  counselor  and  two  college-age  role  models 
who  were  reformed  drug  abusers.  Counselor  functioned  in  same  manner  as  group  3. 

All  of  the  students  (approximately  341)  in  required  health  courses  during  winter  term 
were  randomly  assigned  to  these  four  group  types.  The  control  group  (N  = 33)  was  taken 
from  two  other  health  classes  offered  during  winter  term  in  which  drugs  were  not  discussed. 

Evaluation  criteria  included  gain  in  knowledge,  change  in  attitudes,  and  reduction  in 
drug  abuse  rates,  both  immediately  following  the  study  and  in  a 3-month  followup.  The  three 
major  instruments  used  were  a 41-item  scale  covering  various  issues  (e.g.  legalization  of 
marihuana)  and  a 35-item  health  habits  scale  to  measure  students’  current  involvement. 

Findings: 

1.  All  students  gained  in  knowledge  regardless  of  group  type.  Data  did  not  indicate  any 
differential  impact  by  type  of  group. 

2.  Students  shifted  their  attitudes  in  a liberal  direction.  The  scale  measured  three  separate 
realms:  attitudes  toward  personal  use,  attitudes  toward  drug  abusers,  and  attitudes 
toward  drug  laws.  It  was  impossible  to  tell  if  the  shift  toward  liberalism  was  in  all  realms 
or  perhaps  in  just  one  realm. 
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3.  There  were  no  statistically  significant  changes  in  extent  of  reported  drug  use;  however, 
two  of  the  counseling  approaches  reported  lower  rates  of  drug  use  than  the  discussion 
and  control  groups.  Three  of  the  four  experimental  groups  reduced  their  consumption  of 
alcohol  and  cigarettes. 

4.  The  inclusion  of  models  (i.e.,  drug  experienced  and  non-drug-experienced)  had  no 
differential  impact  in  this  study. 

Swisher,  John  D.,  and  Crawford,  James  L.  “An  evaluation  of  a short-term  drug  education 
program.”  The  School  Counselor,  March  1971.  Pp.  265-272. 

The  program  took  place  in  a private  school  and  was  set  up  and  directed  by  a joint 
student-faculty  committee.  The  three  guidelines  planned  for  this  meeting  were  (a)  The 
faculty  and  administration  would  not  take  part  in  the  program,  (b)  a psychiatric  point  of 
view  should  be  presented,  and  (c)  rehabilitated  drug  users  from  a comparable  socio- 
economic class  should  be  involved. 

The  program  was  given  to  four  high  school  classes.  Each  class  session  varied  slightly. 
The  program  was  offered  on  four  consecutive  Thursday  mornings.  The  ninth  grade  was  set 
up  in  three  1-hour  group  sessions  with  the  same  psychiatrist  conducting  the  session  each 
week.  The  10th  and  11th  grades  had  a 1-hour  group  session,  conducted  by  the  three 
psychiatrists  and  then  split  into  three  smaller  groups  for  the  remaining  2 hours.  The  session 
for  the  12th  graders  was  similar  to  that  of  the  10th  and  11th,  but  involved  several  rehabilitated 
drug  users. 

In  order  to  evaluate  the  impact  of  these  three  approaches  the  following  instruments 
were  employed: 

1.  A 14-item  attitude  scale  that  measured  opinions  about  issues  related  to  drug  abuse, 

2.  A 30-item  knowledge  scale  that  measured  knowledge  about  drugs  in  five  areas,  including 
narcotics,  marihuana,  LSD,  amphetamines,  and  barbiturates, 

3.  A 33-item  behavior  scale  which  assessed  the  health  habits  of  students  and  their  motives 
for  these  habits, 

4.  A 7-item  student  evaluation  scale  which  was  designed  to  assess  student  perceptions  of 
this  particular  program. 

These  scales  were  administered  to  the  students  participating  in  the  drug  program  2 
days  before  the  first  Thursday  session  and  again  1 week  after  the  last  session. 

An  analysis  of  variance  for  the  attitude  scores  yielded  no  significant  differences  between 
or  within  treatments.  The  data  from  the  attitude  tests  revealed  that  the  mean  scores  for  all 
of  the  grade  levels  changed  only  slightly  in  a pro-drug  direction  but  was  not  a statistically 
significant  shift.  However,  the  analysis  of  variance  for  the  knowledge  scale  yielded  significant 
differences  for  the  school  as  a whole  on  pre-  and  posttesting,  as  well  as  significant  differ- 
ences among  the  various  grade  levels  with  the  freshmen  having  the  least  amount  of 
knowledge  and  the  seniors  possessing  the  greatest  amount  of  information. 

In  terms  of  the  students’  perceptions  of  the  drug  education  effort,  the  seniors  rated  this 
program  extremely  high  relative  to  other  schoolwide  programs.  The  sophomores  and  juniors 
who  were  exposed  to  lectures  and  then  group  discussions  rated  their  program  as  being  more 
informative  than  the  other  grade  levels.  The  freshmen  who  were  exposed  to  group  discus- 
sions only  reported  more  discussion  with  their  parents. 

Swisher,  John  D.,  and  Horman,  Richard  E.  “Drug  abuse  prevention.”  The  Journal  of  College 
Student  Personnel,  September  1970.  Pp.  337-341. 

A conference  entitled  “A  Retreat  on  the  Hazards  of  Drug  Abuse”  was  presented  to  the 
students  and  faculty  of  Temple  University.  The  purposes  of  the  conference  were  to  inform 
students,  faculty  and  administrators  about  drugs  and  to  stimulate  their  interest  so  that  they 
would  further  disseminate  relevant  information  to  the  entire  university  population. 
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The  program  included  group  discussions  led  by  a variety  of  experts  (psychiatrists, 
pharmacologists),  in  order  to  evaluate  the  group,  pre-  and  posttesting  were  administered 
seeking  attitude  changes  and  information  gains.  Also  included  was  a followup  questionnaire 
sent  out  6 weeks  later  which  focused  on  the  participants’  activities  in  terms  of  disseminating 
information  to  the  university  at  large.  A control  group  was  also  set  up  from  the  individuals 
who  were  invited  but  for  some  reason  could  not  attend.  Three  evaluation  instruments  were 
developed  for  this  study.  The  evaluation  instruments  were: 

1.  The  Drug  Abuse  Scale  which  contained  achievement  items  focused  on  knowledge  about 
different  types  of  drugs.  The  attitude  subtest  focused  on  issues  related  to  drug  abuse 
(e.g.,  legalization  of  LSD  or  marihuana).  These  items  were  randomly  scattered  among  the 
objective  items  on  the  assumption  that  response  sets  would  be  less  likely  to  occur. 

2.  A Conference  Evaluation  Form  that  assessed  the  participants’  impressions  and  reactions 
to  their  experiences  at  the  retreat. 

3.  The  Drug  Education  Activities  Scale  which  asked  the  participants  to  record  their  behavior 
following  the  conference.  More  than  85  percent  of  the  participants  and  almost  100  percent 
of  the  control  group  returned  this  questionnaire. 

Results: 

“Gain  in  Knowledge” 

The  difference  in  knowledge  gained  was  statistically  significant,  which  indicated  that 
attendance  at  this  type  of  conference  had  an  impact  on  the  participants’  general  level  of 
knowledge  concerning  drugs.  There  were  no  differences  found  when  undergraduates, 
graduates,  and  staff  were  compared. 

“Attitude  Changes” 

The  attitude  data  collected  as  part  of  the  evaluation  revealed  that  at  the  outset  the 
participants  and  controls  generally  had  conservative  attitudes  with  regard  to  the  various 
issues.  All  of  the  statistically  significant  changes  in  attitudes  occurred  among  the  under- 
graduate students.  Data  are  given  in  the  article  to  further  explain  the  exact  attitude  changes 
in  these  students. 

“Activities  Following  the  Retreat” 

The  participants  of  the  conference  reported  being  significantly  more  involved  in  group 
discussions  than  the  control  group.  Participants  also  reported  reading  more  journals  and 
books  than  the  controls. 

“Participants’  Ratings  of  the  Retreat” 

From  data  received,  the  indication  was  that  the  former  drug  users  were  perceived  as 
being  the  most  informative  discussion  leaders.  Based  on  the  results  of  the  evaluation,  the 
following  conclusions  were  developed:  (a)  the  retreat  was  particularly  effective  in  increasing 
the  participant’s  level  of  information  regarding  drugs;  (b)  the  retreat  had  a favorable  impact 
on  the  attitudes  of  undergraduate  students,  particularly  with  regard  to  marihuana;  and  (c)  the 
conference  stimulated  participants  to  further  acquire  and  disseminate  information  related  to 
drug  education. 

Swisher,  John  D.,  and  Horan,  John  J.  “Effective  drug  attitude  change  in  college  students 
via  induced  cognitive  dissonance.”  Pennsylvania  State  University,  Apr.  6,  1972. 

(The  synopsis  below  is  quoted  directly  from  the  article.) 

“The  34  male  and  female  undergraduates  (predominantly  new  freshmen)  who  attended 
a seminar  on  drug  problems  during  orientation  week  at  The  Pennsylvania  State  University 
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were  randomly  assigned  to  an  experimental  and  a control  group.  The  control  and  experi- 
mental groups  were  pre-  and  post-tested  on  the  14  item  attitude  segment  of  the  Drug 
Education  Evaluation  Scales. 

“A  value-preference  inventory  of  the  authors’  own  making  was  also  employed.  This 
instrument  had  a dual  function:  in  the  first  place,  its  18  items  yielded  a score  which  sorted 
the  sample  into  two  groups,  those  preferring  direct  experience  and  those  preferring 
mediated  experiences.  The  instrument  also  served  as  a primary  component  of  the  experi- 
mental treatment.  Upon  receiving  personal  value  information,  students  who  prefer  direct 
experience  could  be  expected  to  become  dissonant  and  thus  more  conservative  in  their 
views  on  drugs.  Since  drugs  are  really  another  way  of  mediating  experience,  attitudes  of 
students  in  the  latter  camp  ought  to  remain  unaffected. 

“Students  who  preferred  direct  experiences,  and  who  were  made  to  feel  dissonant  about 
holding  liberal  drug  views,  showed  considerably  more  conservatism  in  their  attitudes  toward 
drugs  than  similar  students  who  had  not  yet  been  exposed  to  the  experimental  treatment. 

“On  the  other  hand,  students  who  preferred  mediated  experiences  and  who  were  thus 
not  made  to  feel  dissonant  during  the  experimental  treatment  presented  essentially  the  same 
attitudinal  posture  as  similar  students  in  the  control  group.  (Slight  directionality  toward 
liberalism  in  the  latter  instance  should  be  noted  but  interpreted  with  extreme  caution.)” 

Swisher,  John  D.;  Warner,  Richard  W.;  and  Herr,  Edwin  L.  “Experimental  comparison  of  four 
approaches  to  drug  abuse  prevention  among  ninth  and  eleventh  graders.”  Journal  of 
Counseling  Psychology,  Vol.  19,  No.  4,  1972.  Pp.  328-332. 

The  purpose  of  this  study  was  to  investigate  the  effectiveness  of  four  approaches  to 
drug  abuse  prevention  among  secondary  school  youth.  The  four  approaches  were: 

1.  A standard  unit  in  health  classes  focused  on  drug  abuse. 

2.  Relationship  counseling  groups  in  which  the  students  were  allowed  to  explore  the  topic 
of  drug  abuse  in  any  fashion  they  chose.  The  counselor’s  role  was  to  help  the  group 
members  to  accept,  understand,  and  express  themselves.  The  counselor  remained 
neutral  with  regard  to  the  issues  and  was  particularly  careful  not  to  impose  his  opinions 
on  the  group  members.  Participating  counselors  were  given  training  designed  to  heighten 
their  emission  of  empathy,  respect,  and  genuineness. 

3.  Reinforcement  counseling  groups  that  included  a counselor  and  two  college-age  non- 
drug-abusing role  models.  The  role  models  were  selected  in  advance  and  given  an 
orientation  with  regard  to  their  role  in  the  group.  They  facilitated  the  discussions  toward 
reasons  for  not  being  involved  in  drug  abuse.  The  models  were  selected  on  the  basis  that 
although  they  had  not  abused  drugs,  they  knew  the  drug  culture.  The  counselors  were 
to  keep  the  discussion  focused  on  alternatives  and  the  use  of  drugs  and  to  give  positive 
reinforcement  to  statements  made  by  the  students  which  represented  behavior  and 
attitudes  that  would  result  in  not  becoming  involved  with  drugs  at  some  later  time. 

4.  Reinforcement  counseling  groups  that  had  a counselor  and  two  college-age  ex-drug- 
abusing  role  models.  The  role  models  were  similar  to  the  treatment  3 type  models  except 
they  were  chosen  on  the  basis  of  being  former  drug  abusers.  The  counselors  in  this  type 
of  group  functioned  in  the  same  manner  as  the  treatment  3 type  counselor. 

Evaluation 

Three  primary  instruments  were  used  to  identify  and  “tap”  the  three  dimensions  of  the 
problem  of  drug  abuse.  The  three  dimensions  were  given  as  cognitive,  affective,  and 
behavioral.  The  instruments  were: 

1.  A 41-item  achievement  test  which  measured  changes  in  level  of  students’  knowledge 
regarding  the  various  abuses  of  drugs. 
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2.  A 14-item  attitude  scale  which  measured  the  students’  attitudes  regarding  the  various 
issues  (e.g.,  legalization  of  marihuana). 

3.  A 35-item  health  habits  scale  which  assessed  the  students’  current  involvement  with 
drugs,  their  motivations  for  using  drugs,  the  circumstances  in  which  they  abuse  drugs, 
and  their  sources  of  drugs. 

Conclusion 

The  results  of  the  three  tests  given  were: 

1.  Both  9th  and  11th  grades  showed  an  increase  in  knowledge  of  drugs.  The  level  of  gain  in 
drug  knowledge  was  essentially  the  same  in  both  grades. 

2.  There  were  no  significant  changes  in  the  students’  attitudes  toward  drugs  from  pre-  to 
posttesting  occasions. 

3.  There  was  no  significant  change  in  levels  of  drug  use  from  pre-  to  posttesting  occasions. 

Through  the  testing  of  students  it  was  demonstrated  that  knowledge  of  drugs  can  be 
given  as  easily  through  the  traditional  classroom  techniques  (e.g.,  health  class)  as  through 
group  counseling.  They  also  indicated  that  none  of  the  four  approaches  had  any  impact  on 
changing  attitudes  toward  drug  abuse  or  actual  use  of  drugs. 

A second  important  conclusion  resulted  from  the  rating  of  counselor  performance.  All 
group  sessions  were  recorded  and  each  of  the  counselors  was  rated  on  the  Truax  scales  in 
terms  of  criteria  assumed  to  identify  relationship  counseling  and  of  criteria  assumed  to 
identify  reinforcement  counseling.  According  to  the  Truax  scales  of  the  six  counselors,  only 
three  of  them  performed  above  the  minimally  effective  level.  The  other  three  were  below 
that  level. 

Third,  the  study  showed  that  oeer  group  use  was  highly  correlated  with  personal  use 
of  drugs. 

Warner,  Richard  W.,  Jr.,  et  al “Drug  Abuse  Prevention:  A Behavioral  Approach.”  Pennsyl- 
vania State  University,  1972. 

(The  first  paragraph  of  the  synopsis  below  is  quoted  directly  from  the  article.) 

“The  participants  of  the  project  consisted  of  119  ninth-grade  students  who  were  cur- 
rently enrolled  in  a required  health  class  and  3 counselors  who  were  trained  in  conducting 
the  experimental  and  control-procedures.  All  students  were  randomly  assigned  to  one  of 
twelve  counseling  groups.  The  groups  were  then  randomly  assigned  to  one  of  four  treatment 
conditions.  Each  counselor  (also  randomly  assigned)  conducted  one  group  under  each 
experimental  condition.” 

A 14-item  Likert-type  scale  was  administered  anonymously  to  all  groups  at  the  beginning 
and  end  of  the  6-week  experimental  period  to  measure  change  in  attitude  toward  the  use  of 
drugs.  The  study  involved  three  counseling  and  one  no-treatment  control  approaches  as 
follow: 

1.  A behavioral  counseling  group  with  a trained  and  supervised  counselor.  The  counselor 
was  to  have  the  students  consider  viable  alternates  to  the  use  of  drugs. 

2.  A cognitive  dissonance  group  with  a trained  and  supervised  counselor.  This  involves 
showing  pro-drug  attitudes  to  be  inconsistent  with  other  selected  values,  (e.g.  concern 
for  ecology). 

3.  A placebo  group  with  a counselor  who  was  instructed  to  stimulate  a discussion  of  the 
drug  problem.  The  counselor  was  to  listen  to  and  accept  the  comments  of  students  in  a 
nonjudgmental  fashion. 

4.  A control  group  which  participated  in  the  regular  health  unit. 
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Findings: 

The  behavioral  counseling  groups  showed  the  greatest  gain  in  healthy  drug  attitudes; 
however  changes  were  not  significantly  greater  than  those  exhibited  by  the  control  groups. 
Changes  in  the  behavioral  counseling  groups  (approach  1)  were  in  a healthier  direction  than 
those  displayed  by  the  cognitive  dissonance  group  or  the  placebo  group. 

AN  EVALUATION  OF  THE  EFFECT  OF  A VA LUES-ORIENTED  DRUG  ABUSE  EDUCATION 
PROGRAM  USING  THE  RISK-TAKING  ATTITUDE  QUESTIONNAIRE  (1968-1970) 
(ABSTRACT)  1 

Changes  in  the  behavior  of  pupils  in  grades  4-12  of  the  Coronado  Unified  School 
District  were  observed  using  the  Risk-Taking  Attitude  Questionnaire  (RTAQ)  to  evaluate  the 
magnitude  and  direction  of  such  changes.  Experimental  groups  in  the  12th,  6th,  5th  and  4th 
grades  were  compared  to  control  classes  which  were  not  exposed  to  the  drug  abuse  program. 
Experimental  classes  were  given  a values-oriented  drug  abuse  program  which  contained 
both  cognitive  and  affective  components.  “Pre”  and  “post”  administration  of  the  RTAQ  on 
the  same  group  was  done  in  most  cases;  but  in  some,  data  contained  in  previous  studies 
were  used  for  comparison. 

The  hypotheses  of  the  program  were  that  the  amount  of  drug  use  and  other  high-risk 
low-gain  behaviors  would  be  less  for  the  experimental  groups  than  for  the  controls.  In  addi- 
tion, frequencies  of  more  adjustive  behaviors  were  expected  to  be  higher  in  the  experimental 
groups.  A similar  pattern  of  change  was  expected  on  the  rated  risks  and  gains  of  behavior. 
The  experimental  groups  should  have  moved  toward  more  risk  and  less  gain  on  high-risk, 
low-gain  behaviors;  and  toward  less  risk  and  more  gain  on  low-risk,  high-gain  behaviors.  The 
control  groups  were  expected  to  move  in  opposite  directions.  It  was  also  hypothesized  that 
experimental  groups  would  see  more  effectiveness  in  actions  to  control  undesirable  behavior 
by  school,  parents,  etc.,  and  less  effectiveness  for  actions  such  as  dropping  out  of  school 
than  would  the  control  group. 

There  was  some  evidence  in  favor  of  every  hypothesis  with  the  possible  exception  of 
the  “action”  ratings.  Actual  frequency  of  behavior  tended  to  shift  in  the  predicted  direction, 
and  lower  acceptance  (and  use  in  the  upper  age  and  grade  areas)  of  drugs  was  clearly 
evident  in  the  experimental  groups.  Risks  and  gains  attitudes  also  tended  to  move  into  the 
desired  pattern. 

The  clearest  evidence  was  available  for  the  youngest  males  and  on  the  gain  ratings. 
Frequent  differences  based  on  sex  were  also  evident  and  change  points  were  found  at  certain 
ages,  which  indicated  that  particular  types  of  risk  and  gain  were  more  relevant  at  that  age. 
The  gain  of  “feeling  older”  (maturity)  seemed  important  for  12th  grade  males,  while  gains  of 
“acceptance”  (friends)  and  “thrills”  showed  more  change  at  other  ages.  Risk  seemed  to  be 
more  relevant  for  4th  and  6th  grade  males  than  for  any  other  groups. 

EVALUATION  OF  THE  1970-71  CORONADO,  CALIFORNIA,  DRUG  ABUSE  PREVENTION 
PROGRAM  USING  THE  RISK-TAKING  ATTITUDE  QUESTIONNAIRE  (ABSTRACT)  2 

This  paper  examines  the  pre-post  changes  which  took  place  during  the  second  full 
year  of  operation  of  the  Coronado,  California,  “Values”  drug  abuse  prevention  program. 
Experimental  and  control  groups  were  compared  in  the  fourth,  fifth,  and  sixth  grades  and 
experimental  groups  only  in  the  junior  and  senior  high  schools.  For  various  reasons,  it  was 
not  possible  to  carry  out  the  original  research  design,  and  comparisons  of  the  experimental- 
control  categories  were  meaningless  in  the  fourth  through  sixth  grades.  Analysis  of  the  data 
based  on  these  categories  showed  little  systematic  change. 


1 This  abstract  was  written  by  Richard  E.  Carney,  Ph.D.,  Educators  Assistance  Institute,  Santa  Monica,  California. 

-This  abstract  was  written  by  Richard  E.  Carney,  Ph.D.,  Educators  Assistance  Institute,  Santa  Monica,  California. 
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When  individual  classes  were  ranked  on  the  degree  of  training  for  and  use  of  program 
materials  and  on  the  amount  of  “values-orientation”  of  the  teacher,  clear  indication  of 
program  effectiveness  was  found.  Males  in  the  most  values-oriented  classes  tended  to 
change  to  lower  drug  use  and  greater  participation  in  positive  behaviors  such  as  working 
hard;  however,  girls  showed  little  change. 

Moderate  success  was  obtained  in  cross-validating  prediction  equations  for  behaviors 
using  attitude  ratings  as  predictors.  Some  evidence  for  a positive  effect  of  the  program  on 
attitude  changes  was  found  at  the  secondary  level,  but  these  changes  were  difficult  to 
evaluate  because  of  the  lack  of  control  groups. 

When  attitude  changes  were  examined  in  the  fourth  through  sixth  grades,  some  lowered 
predicted  involvement  by  values  classes  of  males  in  “drug-abuse”  behaviors  was  found. 
Females  in  the  values  classes  tended  to  have  higher  predicted  scores.  Comparisons  of 
changes  in  perceived  risk  and  gain  showed  strong  selective  movement  by  males  in  values 
classes  toward  increased  ratings  of  risk  and  lowered  ratings  of  gain  for  “drug-abuse” 
behaviors.  Females,  again,  showed  an  opposite  pattern. 

REPORT  ON  THE  USE  OF  THE  RTAVI  TO  EVALUATE  VALUES,  ATTITUDES,  AND 
BEHAVIORS  IN  THE  MADISON  SCHOOL  DISTRICT,  PHOENIX,  ARIZONA  (1972) 
(ABSTRACT)  3 

Groups  of  82  fourth  grade  and  75  seventh  grade  pupils  from  Phoenix,  Arizona  were 
included  in  a “Valuing”  drug-abuse  prevention  education  program  during  the  1971-72  school 
year.  Control  groups  of  18  fourth  grade  and  19  seventh  grade  pupils  were  used  for  com- 
parison of  pre-post  changes  that  might  be  attributable  to  the  “Valuing”  program.  The  Risk- 
Taking  Attitude-Values  Inventory  Elementary  form  was  used  “pre”  and  “post”  to  measure 
five  clusters  of  attitudes  and  one  cluster  of  15  reported  behaviors.  Within  each  cluster  the 
items  were  arranged  from  lowest  to  highest  average  and  it  was  hypothesized  that  an  ideal 
outcome  would  produce  progressively  greater  increases  in  post  averages  toward  the  high 
end  of  the  set  of  items.  That  is,  the  valuing  groups  should  hold  constant  or  decrease  low 
frequency  (high  risk-low  gain)  behaviors  such  as  drug  abuse  and  increase  high  frequency 
behaviors  such  as  working  and  art  relative  to  the  control  groups. 

Out  of  24  possibilities  (two  groups,  two  sexes,  six  clusters),  13  were  definitely  in  favor 
of  the  ideal  outcome,  four  slightly  in  favor,  six  showed  no  difference  and  in  one  case  there 
was  a result  slightly  against  the  valuing  program.  The  most  favorable  results  were  obtained 
on  the  Importance  of  Values,  Ways  of  Changing  Behavior  and  Frequency  of  Behavior 
clusters.  Relatively  little  favorable  change  was  found  on  the  Utility  and  Expectancy  of 
Behavior  ratings. 

VALUING  AND  DRUGS— FINAL  1971-72  REPORT  ON  THE  TEMPE,  ARIZONA  DRUG 
ABUSE  PREVENTION  PROGRAM  (ABSTRACT)  4 

Pre-post  changes  in  averages  on  the  Risk-Taking  Attitude-Values  Inventory  were  exam- 
ined for  the  fourth,  sixth,  and  eighth  grades  in  the  Tempe,  Arizona  School  District  #3. 
Various  combinations  of  Valuing  (V)  and  Drug  fact  (D)  programs  were  compared  to  control 
groups  who  received  no  special  attention.  Beth  changes  in  reported  raw  scores  and  in 
predicted  frequencies  of  behavior  were  analyzed  by  ANOVA  techniques.  Changes  in  raw 
score  Values  Importance  and  Values  Nearness  ratings  were  also  tested  by  ANOVA. 

Clearly  favorable  outcomes  in  both  raw  and  predicted  frequencies  of  behavior  were 
found  for  the  eighth  grade  males  in  the  V and  D + V groups.  Drug  use  was  less  than 
controls  and  “good”  behaviors  such  as  working  and  sports  were  higher.  Similar  but  less 
clear-cut  findings  were  obtained  for  the  eighth  and  sixth  grade  females  in  the  V group  with 

“This  abstract  was  written  by  Richard  E.  Carney,  Ph.D.,  Educators  Assistance  Institute,  Santa  Monica,  California. 

4 This  abstract  was  written  by  Richard  E.  Carney,  Ph.D.,  Educators  Assistance  Institute,  Santa  Monica,  California. 
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more  negative  results  in  V + D and  D groups.  Little  consistent  change  was  found  for  any 
group  of  sixth  grade  males  and  for  the  V + D groups  of  fourth  grade  males  and  females. 

Some  favorable  program  effect  on  values  ratings  was  found  for  every  grade  and  sex 
group.  The  differences  from  controls  tended  to  be  small  and  inconsistent  from  one  grade, 
sex,  and  program  group  to  another. 
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RESOURCES 


The  resources  listed  here  provide  guidelines  for  evaluation  or  give  examples  of  instru- 
ments, measures,  or  strategies  that  have  been  used  in  evaluation.  The  reader  will  have  to 
judge  for  himself  the  validity  of  the  instruments  or  measures  and  their  usefulness  in  evalu- 
ating his  program. 

Accountability  in  Drug  Education:  A Model  for  Evaluation.  Washington,  D.C.  The  Drug  Abuse 
Council,  1973. 

Drug  Abuse  Research  Instrument  Inventory.  Fourth  Edition,  Cambridge,  Massachusetts. 
Social  Systems  Analysts. 

Guidelines  for  Evaluation  of  Continuing  Education  Programs  in  Mental  Health.  Rockville, 
Maryland:  National  Clearinghouse  for  Mental  Health  Information,  National  Institute  of 
Mental  Health. 

Measures  Pertaining  to  Health  Education:  Drugs;  An  Annotated  Bibliography.  Princeton, 

New  Jersey:  TM  Reports  No.  9,  ERIC  Clearinghouse  on  Tests,  Measurements,  and 
Evaluation,  Educational  Testing  Service. 

Richards,  Louise  G.  “Evaluation  in  Drug  Education.”  School  Health  Review.  Vol.  2,  No.  3, 
September  1971,  (This  article  reviews  several  evaluation  studies.) 

Webb,  Eugene  J.;  Campbell,  Donald  T.;  Schwartz,  Richard  D.;  and  Sechrest,  Lee.  Unobtrusive 
Measures:  Nonreactive  Research  in  the  Social  Sciences.  Chicago:  Rand  McNally,  1966. 


U.S.  GOVERNMENT  PRINTING  OFFICE:  1976  0 — 220-729 


32 


DATF 


m LIBRARY 


Amazing  Help. 

http://nihlibrary.nih.gov 


dd  d/w)?? 


1 0 Center  Drive 
Bethesda,  MD  20892-1150 
301-496-1080 


496  00005  4745 


DHEW  Publication  No.  (ADM)  76-234 
Formerly  DHEW  Publication  No.  (ADM)  75-234 
Printed  1975  Reprinted  1976 


